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to 

Directorate  of  Logistics  Management 
Systems  Requirements  (XRB) 
DCS/Plans  and  Programs 
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INTRODUCTION 


Under  Contract  No.  F33600-81-C-0613 ,  Battelle's  Columbus  Laboratories 
has  been  supporting  AFLC/XRB  in  the  application  of  the  long-range  LMS  planning 
system,  developed  under  Contract  No.  F33600-80-C-0414 .  The  Planning  Model  pro¬ 
vided  a  guide  for  the  planning  activities  conducted  under  Contract  0613.  The 
Battelle  responsibility  was  to  monitor  planning  activities  and  to  make  recom¬ 
mendations  for  improvement  of  the  planning  model. 

It  is  the  purpose  of  this  document  to  comment  on  the  success  of  the 
planning  model  as  it  was  applied  to  the  Improvement  Process. 

This  report,  a  Lessons  Learned  document,  is  presented  in  two  parts. 
The  first  covers  support  activities  to  Level  II  Improvement  planning  from  its 
initiation  in  September  1981  through  the  initial  planning  meeting  held  at  San 
Antonio  ALC  on  November  3-5,  1981. 

The  second  part  addresses  the  support  provided  to  planning  sessions 
held  via  Teleteach  in  December  1981  and  January  1982. 


L 
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PART  I:  INITIAL  SUPPORT  ACTIVITIES 


Preparation  Procedures 

The  initial  activity  involved  preparation  of  a  plan  for  accomplishing 
the  first  two  planning  steps.  These  planning  steps  were  specifically  directed 
toward  identifying 

(a)  What  Exists  Today,  and 

(b)  Needed  Capabilities 

for  long-range  LMS  development  in  support  of  the  Improvement  Process.  The 
required  activities  were  identified  and  tentative  schedules  prepared  independ¬ 
ently  by  both  Battelle  and  XRB  and  then  reconciled,  with  responsibilities  for 
preparation  of  planning  materials  mutually  determined. 

The  three  major  activities  involved  (a)  developing  a  strawman  descrip¬ 
tion  of  the  Improvement  Process,  its  LAGs  and  functions  as  they  exist  today  and 
how  they  might  look  in  the  future,  (b)  making  preparations  for  the  first  plan¬ 
ning  meeting,  and  (c)  conducting  the  first  planning  meeting. 

Under  activity  (a) ,  Battelle  developed  a  background  package  for  mis¬ 
sion  planners  at  Headquarters  AFLC  to  use  in  preparing  the  strawman.  This 
included  a  brief  introduction  to  long-range  LMS  planning  and  a  description  of 
the  steps  to  be  followed  in  developing  the  strawman.  Several  attachments  were 
contained  in  the  package  including  descriptions  of  the  Improvement  Process 
related  to  the  extract  chart  and  the  Mission  Support  Document  (MSD);  descriptions 
of  LAGs  29,  30,  31,  and  32;  Command  shortfalls  related  to  the  Improvement  Pro¬ 
cess;  and  extracts  from  AFLC  Command  Level  Guidance  for  LMS  Planning.  Also 
included  were  sample  work  sheets  for  developing  descriptions  of  each  function 
including  decisions,  responsible  organizations,  input  information  and  its  sources, 
and  output  information  and  its  destinations. 

Development  of  the  strawman  functional  descriptions  for  both  the  cur¬ 
rent  and  future  systems  as  well  as  definition  of  current  and  future  shortfalls 
was  performed  by  mission  personnel  from  LOL  and  LOE.  Meetings  were  held  with 
each  group  to  review  the  background  materials  and  to  begin  development  of  the 
strawman.  Both  XRB  and  Battelle  personnel  participated  in  these  meetings.  The 
mission  planners  then  proceeded  to  develop  the  strawman  under  the  schedule  laid 
out  by  M.  Rhotie/XRB  and  under  her  guidance  when  questions  arose.  Final  format- 
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ting  of  the  strawman  materials,  typing,  and  preparation  of  vugraphs  was  per¬ 
formed  by  XRB  and  reviewed  by  Battelle. 

Arrangements  for  the  meeting  at  San  Antonio  ALC  on  November  3-5,  1981 
with  representatives  from  both  Oklahoma  City  and  San  Antonio  ALCs  were  made  by 
XRB  and  coordinated  through  LOM  at  Headquarters.  Attendees  were  jointly  agreed 
to  by  the  ALCs,  XRB,  and  LO. 

The  meeting  schedule,  presentation  format,  and  selection  of  facili¬ 
tators  and  recorders  were  decided  jointly  by  XRB  and  Battelle. 

Planning  Content 

Input 


The  basic  planning  materials  prepared  for  the  San  Antonio  meeting 
consisted  of  a  drawing  representing  each  function  as  it  exists  today  and  a 
drawing  as  it  should  look  in  the  future.  On  each  drawing  was  shown  the  function 
block  giving  the  function  name,  the  organizations  responsible  for  performing 
the  function  and  the  major  decisions  made  in  performing  the  function..  Also 
shown  were  the  major  information  inputs  to  the  function  and  their  sources,  and 
the  major  information  outputs  and  their  destinations.  Associated  with  each 
current  function  drawing  was  a  list  of  current  shortfalls,  and  with  each  future 
function  drawing,  a  list  of  future  shortfalls. 

Only  one  LAG,  LAG  32  Tech  Order  Management,  had  more  than  one  function. 
For  this  LAG  a  second  type  of  drawing  was  provided  showing  information  flows 
among  the  three  functions  involved.  Separate  drawings  were  provided  for  both 
the  current  and  future  versions  of  the  LAG. 

Finally,  drawings  were  provided  for  both  the  current  and  future  versions 
of  the  entire  Improvement  Process  showing  major  information  flows  between  the  LAGs 
that  constitute  the  process. 

All  planning  materials  were  developed  by  appropriate  functional  experts 
in  LO  who  followed  the  formats  provided  by  XRB  and  Battelle.  Final  preparation 
and  reproduction  of  the  planning  materials  was  performed  by  XRB. 
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Facilities 


The  San  Antonio  planning  meeting  was  held  in  the  XR  conference  room 
at  San  Antonio  ALC.  The  room  was  equipped  with  a  screen  and  dual  vugraph  pro¬ 
jectors,  and  a  central  table. 

Process 


The  meeting  was  introduced  by  Mel  Lammers/XRB  who  reviewed  the  intent 
of  long-range  LMS  planning  based  on  institutionalizing  the  planning  process  and 
addressing  LMS  renewal  on  a  modular  basis  to  make  the  planning  manageable,  to 
reduce  the  overall  risk,  and  to  spread  the  planning  investment. 

This  introduction  was  followed  by  a  brief  introduction  to  the  LMS 
planning  process  given  by  Doug  Hill  of  Battelle. 

The  balance  of  the  meeting  basically  followed  the  schedule  shown  in 
Figure  1.  The  planning  sessions  were  facilitated  for  the  most  part  by  M.  Rhone/ 
XRB .  HQ  AFLC/LO  staff  acted  as  resource  persons  to  clarify  how  the  planning 
materials  were  developed  and  the  underlying  rationale.  The  remainder  of  the 
attendees  shown  in  Figure  2  performed  the  actual  planning. 

Output 


The  planning  was  conducted  by  reviewing  and  discussing  the  planning 
materials  and  modifying  or  augmenting  them  as  required.  Changes  and  additions 
were  noted  on  vugraphs  of  the  planning  materials  during  the  discussions.  These 
materials  were  then  "cleaned  up"  and  distributed  to  the  meeting  attendees. 

During  the  final  afternoon  of  the  planning  meeting,  the  discussions 
centered  on  the  development  of  alternative  approaches  to  overcoming  the  identi¬ 
fied  shortfalls,  both  current  and  future.  This  session  was  intended  to  intro¬ 
duce  the  concept  of  alternative  approaches  to  the  planning  participants.  The 
next  step  was  for  them  to  generate,  prior  to  the  next  planning  meeting,  alter¬ 
native  approaches  to  the  identified  shortfalls  contained  in  the  "cleaned  up" 
planning  materials  that  were  distributed  to  them.  These  alternative  approaches 
were  then  to  be  screened  against  standard  Air  Force  policies  and  good  business 
practices . 
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Finally,  those  approaches  that  passed  the  screen  were  to  be  evaluated 
against  a  variety  of  criteria  such  as  cost,  timeliness,  consistency  with  other 
projects,  and  benefit  to  the  Air  Force. 


Finally,  those  approaches  that  passed  the  screen  were  to  be  evaluated 
against  a  variety  of  criteria  such  as  cost,  timeliness,  consistency  with  other 
projects,  and  benefit  to  the  Air  Force. 
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•  Recommended  Changes 

As  a  result  of  the  preparation  for  and  conduct  of  the  first  two  steps 
of  the  planning  process,  a  number  of  observations  have  been  made  that  relate  to 
improving  the  effectiveness  and  efficiency  of  carrying  out  those  steps  for 
other  processes  and  perspectives.  In  the  following  sections  the  observations 
have  been  grouped  and  explicit  recommendations  made  for  improving  the  planning 
process . 


Planning  Materials  and  Meeting  Preparations 

1.  The  principles  expressed  by  M.  Lammers/XRB,  namely: 

•  Logistics  requirements  must  drive  hardware 

acquisitions,  and  • 

•  Flexibility  is  paramount  because  of  uncertainty 
of  the  future 

are  important  and  need  to  be  given  high  visibility 
in  the  package  of  introductory  materials  provided 
to  both  HQ  and  ALC  planners. 

2.  There  was  some  redundancy  between  the  introductory 
presentations  by  M.  Lammers/XRB  and  D.  Hill/Battelle . 
These  presentations  should  be  compared  prior  to 
presentation  to  avoid  such  redundancy  in  the  future. 

3.  The  rationale  behind  the  development  of  the  Process 
charts  and  LAG  charts  was  not  a  part  of  the  planning 
materials.  The  rationale  should  be  clearly  described 
and  included  in  the  package  of  planning  materials. 

4.  The  information  flows  to  and  from  functions  were  not 
labeled  as  to  whether  their  origin  or  destination 
was  outside  the  LAG  and/or  Process.  Such  labeling 
should  be  included  on  the  strawroan  function  charts 
to  facilitate  the  development  of  IAG  charts,  and  the 

.  process  chart. 
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5.  In  some  cases,  an  information  output  from  a  function  that 
flowed  to  another  function  in  the  same  LAG,  or  to  another 
function  in  a  different  LAG  in  the  Improvement  Process, 
was  labeled  differently  than  the  corresponding  information 
input.  Such  inconsistency  makes  the  formation  of  LAG  charts 
and  Process  charts  difficult.  It  is  recommended  that  the 
inconsistencies  be  resolved  prior  to  the  planning  meeting. 
This  would  probably  require  that  the  functional  planners 
develop  the  LAG  and  process  charts. 

6.  The  overall  complexity  of  interconnecting  information 
flows  across  all  processes  and  perspectives  overwhelm  the 
people  that  recognize  the  complexity.  An  explanation 

of  how  these  interconnections  are  to  be  handled  through 
the  configuration  control  procedure  should  be  developed 
and  made  available  to  functional  area  planners. 

7.  The  futurity  document  was  misinterpreted  as  a  constraining 
set  of  factors  that,  had  to  be  addressed.  This  material  has 
to  be  presented  as  a  thought  stimulator  to  get  planners  to 
consider  future  requirements  -  not  as  a  set  of  constraining 
requirements  that  have  to  be  addressed. 

8.  The  ALC  planners,  and  to  some  extent  the  HQ  functional  area 
planners  tend  to  think  about  "how  they  go  about  solving  a 
problem".  Their  orientation  should  be  toward  "what  informa¬ 
tion  is  needed  to  make  the  decisions  associated  with  a 
function".  The  solution  to  this  problem  would  seem  to  be 
working  closely  with  the  functional  area  planners  during 
consideration  of  the  first  couple  of  functions  and  a 
tutorial  "walk-through"  session  with  ALC  planners  on  at  least 
one  function. 

9.  Some  shortfalls  were  not  sufficiently  explicit  -  at  least  for 
the  ALC  planners.  They  should  be  defined  relative  to  a 
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performance  standard  (i.e.,  what  is  satisfactory)  that  has 
a  meaningful  measure  associated  with  it. 

10.  "Shortfall"  was  considered  to  be  a  poor  term  for  use  with 

the  future  version  of  functions.  "Future  requirement"  is  the 
preferred  term. 

Selection  of  Planning  Personnel 

The  following  observations  are  made  without  comment  but  lead  to  a 
major  recommendation  at  the  end  of  this  section. 

1.  The  ALC  people  had  difficulty  relating  to  the  function 
diagrams,  decisions,  and  shortfalls.  These  factors 
are  at  a  considerably  higher  level  of  abstraction  than 
these  people  normally  consider. 

2.  The  ALC  people  were  not  adequately  familiar  with  the 
planning  materials  forwarded  to  them  prior  to  the  meeting. 
Many  had  simply  not  studied  the  material  enough  to 
thoroughly  understand  it. 

3.  The  presence  of  HQ  AFLC  planners  at  San  Antonio  was  neces¬ 
sary  to  explain  the  strawman.  Nevertheless,  their  presence 
was  intimidating  to  some  of  the  ALC  people  as  exhibited  at 
times  by  their  deference  to  the  HQ  position. 

4.  It  was  particularly  difficult  to  get  the  ALC  planners  to 
divorce  themselves  from  consideration  of  the  existing 
DSDs  and  focus  on  the  functions. 

5.  The  ALC  people  seemed  to  have  particular  difficulty  in 
putting  themselves  in  a  frame  of  mind  to  consider 
future  requirements. 

6.  The  ALC  people  knew  their  systems  and  detailed  functions 
extremely  well. 


7.  The  "strawman"  was  changed  relatively  little  as  a  result 
of  the  San  Antonio  meeting.  It  is  likely  that  similar  or 
even  improved  results  might  have  been  obtained  through 
application  of  the  same  effort  by  HQ  functional  area  planners. 

As  a  result  of  the  above  observations,  it  appears  that  Level  II 
planning  should  be  carried  out  only  by  HQ  functional  area  planners.  ALC  planner 
could  carry  the  primary  responsibility  for  Level  III  planning  for  those  func¬ 
tions  for  which  they  are  responsible.  Functions  that  are  a  HQ  responsibility 
could  be  the  responsibility  of  HQ  personnel  insofar  as  Level  III  planning  is 
concerned . 


PART  I I :  SUPPORT  TO  TELETEACH  SESSIONS 


Preparation  Procedures 

During  November  and  early  December  1981,  there  were  a  series  of 
meetings  between  XR£,  LO,  and  Battelle  to  decide  on  the  next  steps  in  light  of 
the  San  Antonio  experience  and  in  view  of  extremely  limited  travel  funds. 

A  part  of  the  preparations  centered  on  developing  an  understanding  of 
the  LMS  modernization  program  within  LO  at  HQ  AFLC  and  within  MM  at  the  ALCs. 

In  this  regard  a  series  of  tutorial  briefings  were  developed  describing  the 
LMS  modernization  program  and  the  planning  methods  to  be  used.  These  briefings 
were  presented  on  December  11  and  14  at  HQ  AFLC. 

LO  felt  that  all  ALCs  should  be  involved  with  Improvement  LMS  planning. 
With  this  guidance  and  the  travel  limitations  it  was  decided  to  use  the 
TeleTeach  facilities  at  AFIT  and  at  each  of  the  ALCs  to  conduct  the  next  plan¬ 
ning  sessions.  This  had  the  advantage  of  allowing  participation  of  all  ALCs 
and  the  involvement  of  more  people  at  each  ALC  although  the  introduction  of 
new  people  implied  the  need  for  tutorial  material  being  presented. 

The  TeleTeach  sessions  were  scheduled  beginning  December  15  with 
subsequent  sessions  scheduled  on  December  16-18  and  January  5-8  and  January  11 . 

In  preparation  for  the  December  15  meeting,  Battelle  and  XRB  developed 
agendas  for  the  meetings  as  well  as  tutorial  material.  Packages  of  planning 
outputs  that  resulted  from  the  San  Antonio  meeting  for  each  of  the  three  LAGs 
were  also  prepared  and  distributed  to  all  five  ALCs. 

Planning  Content 


Input 


The  planning  materials  provided  to  planners  at  each  ALC  included 
introductory  briefing  materials  on  LMS  Requirements  Determination,  on  the  three- 
level  planning  process  for  identifying  LMS  requirements  ,  and  on  the  output  of  the 
San  Antonio  meeting. 
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A  background  package  consisting  of  the  following  14  items  was  provided 
to  each  participant  : 

1.  Extract  Chart 

2.  Long-Range  LMS  Planning:  Improvement  Process: 

Level  II,  Block  1. 

3.  Long-Range  LMS  Planning  Paper 

4.  Improvement  Process  (Extract  Chart  Description) 

5.  Improvement  Process  (MSD) 

6.  LAG  29  -  Monitor  Field  Performance 

7.  LAG  30  -  Materiel  Performance  Analysis 

8.  LAG  59  -  Monitor  Product  Performance 

9.  LAG  31  -  Materiel  Deficiency  Report  Control 

10.  LAG  32  -  T.O.  Management 

11.  Blank  Worksheets 

12.  Improvement  Process  Shortfall  Extraction 

13.  "Futurity"  Document 

14.  Improvement  Strawman  (Revised). 

In  addition,  a  guidance  document  on  the  preparation  of  alternative 
approaches  was  provided. 

Facilities 


The  December  15  and  16,  1981  TeleTeach  Sessions  were  held  at  the 
AFIT  TeleTeach  room  at  WPAFB  at  the  TeleTeach  Facilities  at  each  of  the  ALCs. 

The  AFIT  facility  was  equipped  with  individual  "push-to-talk" 
desk  microphones  as  well  as  the  electronic  blackboards,  audio  mixing  equipment, 
and  speakers. 

Process 


The  first  Improvement  Planning  Session  conducted  through  TeleTeach 
began  on  December  15,  1981  with  HQ  AFLC/LO  and  XR,  and  representatives  from  MM 
and  XR  at  all  ALCs  participating.  While  necessary  planning  materials  had  been 
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hand  carried  to  the  ALCs  on  December  11,  1981,  they  had  not  been  adequately 
reproduced  at  the  ALCs  and  distributed  to  participants.  This  neccessitated 
some  adjustment  in  the  presentation  approach  and  schedule  as  described  below. 

The  first  activity  was  a  "sign-in"  process  by  all  ALC  participants  to 
find  out  who  was  present  and  verify  current  operation  of  the  electronic 
blackboard  and  audio  links.  The  system  operated  satisfactorily  apart  from 
some  occasional  lack  of  clarity  with  the  audio  link. 

After  initial  remarks  by  M.  Lammers  (XRBT),  Dr.  Hill  of  Battelle 
presented  the  "Briefing  on  LMS  Requirements  Planning"  that  had  been  presented  to 
the  DMMs  and  LO  on  December  11  and  to  PM  on  December  14  by  K.  Miller  of  Battelle. 
Because  hard  copy  of  the  VuGraphs  was  not  available  to  the  ALC  participants, 
they  had  to  be  copied  on  the  electronic  blackboard.  Consequently  the  twenty 
minute  briefing  lasted  more  than  one  and  one-half  hours.  This  may  not  have 
been  all  bad  however  since  the  material  was  totally  new  to  most  ALC  participants. 
What  was  originally  a  briefing  became  much  more  of  a  tutorial  session. 

After  a  short  lunch  break,  LO  personnel  took  some  time  to  confirm  the 
organization  of  planning  groups  to  address  each  LAG.  M.  Lammers  then  presented 
the  revised  schedule  for  the  balance  of  December  15  and  for  December  16. 

D.  Hill  then  presented  the  LMS  Requirements  Planning  backup  VuGraphs 
that  provide  further  explanation  of  the  eight  planning  steps  and  provide 
sample  outputs  taken  from  the  Level  III  Maintenance  LAG  50  planning  output.  This 
presentation  moved  fairly  quickly  since  by  this  time  that  set  of  VuGraphs  had 
been  reproduced  and  distributed  to  all  participants. 

On  December  16,  1981  Mel  Lammers  opened  the  planning  session  with  a 
brief  review  of  processes  and  LAGs  for  the  benefit  of  some  new  participants 
at  the  ALCs.  Dr.  Hill  then  reviewed  the  contents  of  the  planning  package  that 
had  been  used  by  LO  to  develop  the  strawman  LAGs  and  by  the  participants 
at  the  San  Antonio  meeting  in  November.  This  was  followed  by  a  review  of 
LAG  31  led  by  George  Kawanishi  (LOLM) . 

After  the  LAG  31  discussion,  LAG  32  was  briefly  discussed.  Then 
D.  Hill  discussed  the  concept  of  developing  and  structuring  change  objectives  as 
a  step  leading  to  the  definition  of  alternative  approaches.  LAG  32  current 
shortfalls  were  used  in  an  example. 
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After  the  discussion  on  change  objectives  C.  Gridley  (LOEP)  led 
a  discussion  of  LAG  36.  This  again  resulted  in  a  good  discussion.  Finally 
G.  Kawanishi  gave  the  planners  their  assignments  and  reconfirmed  the  TeleTeach 
Sessions  on  January  5,  1982  for  LAG  32,  on  January  6,  1982  for  LAG  31  and  on 
January  7  and  January  8,  1982  for  LAG  76. 

Subsequent  to  the  December  16  Session,  Battelle  did  not  participate 
in  the  TeleTeach  Sessions.  Battelle  did  provide  guidance  to  XRB  analysts 
in  preparing  for  TeleTeach  Sessions,  particularly  in  the  area  of  defining 
shortfalls,  change  objectives,  and  alternative  approaches. 

Battelle  also  assisted  in  two  meetings  held  on  February  10,  1982 
designed  to  introduce  and  demonstrate  the  application  of  evaluation  criteria 
to  the  prioritization  of  alternative  approaches  to  attaining  change  objectives. 

The  first  meeting  addressed  change  objectives  and  alternative  approaches 
developed  for  Lag  31:  Material  Deficiency  Reporting  (MDR).  Some  time  was 
spent  discussing  the  evaluation  criteria  that  the  MDR  group  had  developed 
and  it  was  decided  to  use  an  evaluation  scale  of  5  (actually  from  —  to  ++  to 
avoid  numbers).  Alternative  approaches  were  evaluated  for  two  of  the  five  MDR 
change  objectives.  The  procedure  worked  quite  well  and  the  group  decided  that 
they  would  be  able  to  proceed  with  the  evaluations  on  their  own. 

In  the  second  meeting  XRB  and  Battelle  analysts  met  with  the  Lag  32: 

Tech  Order  Management  group.  Some  time  was  spent  discussing  the  change  objectives 
and  alternative  approaches  that  had  been  developed  for  this  Lag  by  Oklahoma 
City  ALC  personnel.  Some  inconsistencies  were  identified  and  G.  Kawanishi 
decided  that  his  HQ  people  should  review  the  material  in  detail  before  going 
through  the  evaluation  procedure.  Some  time  was  spent  discussing  the  evaluation 
criteria  and  results  of  the  morning  session.  Kawanishi  seemed  to  think  the 
procedure  was  acceptable. 

These  meetings  on  February  10,  1982  were  the  last  substantive  support 
that  Battelle  provided  to  Level  II  Improvement  Planning. 

Output 


The  first  time  that  the  ALC  participants  had  an  opportunity  to  make 
additions  and  changes  to  the  previously  generated  planning  results  was  after 
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G.  Kawanishi's  review  of  LAG  31  on  December  16.  As  a  result  of  the  LAG  31 
discussion  some  new  information  outputs  and  destinations  were  added  and  an 
error  in  the  symbology  was  identified. 

Similarly,  good  discussion  and  some  relatively  minor  improvements  to 
the  planning  results  were  made  during  the  TeleTeach  session. 

Subsequent  to  the  TeleTeach  sessions  on  December  15  and  16,  Headquarters 
AFLC  and  ALC  staff  significantly  reworked  the  planning  outputs  with  an  emphasis 
on  identifying  current  and  future  shortfalls  and  the  approaches  needed  to  over¬ 
come  these  shortfalls. 


Recommendations 


This  set  of  recommendations  reflects  a  somewhat  limited  role  by  Battelle 
in  the  TeleTeach  and  subsequent  LMS  planning  meetings  for  the  Improvement  Process. 
There  are  three  areas  addressed.  The  first  involves  some  observations  on  use 
of  the  TeleTeach  technology.  The  second  involves  the  preparation  of  materials 
to  bring  new  people  into  the  planning  process.  Finally,  there  are  a  few  comments 
on  developing  evaluations  of  alternative  approaches. 

TeleTeach 


The  TeleTeach  technology  is  effective  when  it  is  working  properly  and 
when  appropriate  planning  materials  have  been  prepared  and  delivered  to  partici¬ 
pants.  In  our  experience,  there  were  technical  difficulties  in  keeping  both  the 
audio  and  graphics  capabilities  operational  at  all  five  ALCs  and  at  HQ  AFLC. 

It  is  disruptive  to  everyone  on  the  network  when  one  terminal  loses  audio  and / 
or  graphics  capability.  Everyone  has  to  wait  while  technicians  attempt  to  get 
the  network  patches  back  together. 

On  a  more  positive  note,  it  was  found  that  TeleTeach  allowed  for  wide 
participation  in  the  LMS  planning,  and  for  bringing  in  specialists  in  a  particular 
functional  area  on  an  "as  needed"  basis. 

The  TeleTeach  tends  to  focus  the  group  on  whomever  is  speaking  or 
writing.  Corner  discussions  seem  to  be  much  less  of  a  problem  than  they  would 
otherwise  be  in  a  meeting  with  the  number  of  participants  involved  in  the 
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Improvement  planning  sessions.  This  large  number  of  people  does  make  it 
important  to  establish  and  enforce  a  speaker  protocol  at  the  beginning  of  each 
session.  It  is  simply  a  matter  of  having  each  speaker  giving  his  name  and 
affiliation/location  each  time  he  speaks  on  the  network.  But  it  is  important 
that  the  speaker's  identify  be  established  each  time  the  person  speaks  so  that 
everyone  can  keep  track  of  the  discussion. 

Using  TeleTeach,  it  was  found  that  changes  to  previously  prepared 
planning  materials  could  be  generated.  Some  difficulty  was  experienced  in 
capturing  and  documenting  shortfalls  as  they  were  identified.  It  is  difficult 
to  judge  the  relative  effectiveness  of  this  approach  when  compared  to  a  face- 
to-face  meeting  with  a  large  group  and  all  of  its  attendant  problems,  but  the 
general  feeling  is  that  TeleTdach  provided  an  adequate  medium  of  communications 
when  it  was  working  properly. 


Preparation  of  Materials 


As  indicated  previously,  the  preparation  of  tutorial  materials  becomes 
particularly  important  in  the  TeleTeach  environment.  TeleTeach  promotes 
considerable  flexibility  in  involving  many  different  people  in  the  planning 
process  who  may  be  totally  unfamiliar  with  the  LMS  Modernization  Program  and 
with  the  methodology  for  LMS  Requirements  Determination.  The  materials  prepared 
to  help  communicate  the  essence  of  the  Modernization  program  and  Requirements 
Determination  Process  were  simply  not  available  in  time  for  people  to  read  them 
before  the  first  meetings  in  December.  But  even  if  they  had  been  available, 
it  is  not  clear  that  they  would  have  been  understood  by  the  participants.  At¬ 
taining  understanding  takes  time.  Even  with  the  availability  of  improved 
materials  such  as  contained  in  the  three-volume  "Handbook  for  Logistics  Manage¬ 
ment  Systems  (LMS)  Requirements  Determination  Planning  Process"  it  will  require 
the  commitment  of  significant  amounts  of  staff  time  for  the  participants  to 
develop  the  understanding  necessary  to  facilitate  their  effective  particpation 
as  LMS  planners.  This  requirement  must  be  recognized  and  satisfied  as  well  as 
the  requirement  for  direct  involvement  in  planning  activities. 


Evaluation  of  Alternative  Approaches 


Finally,  the  area  of  evaluation  of  alternative  approaches  deserves 
comment.  The  structure  originally  proposed  by  Battelle  involved  the  identifica¬ 
tion  of  shortfalls,  grouping  the  shortfalls  under  change  objectives,  and  then 
developing  and  evaluating  alternative  approaches  to  satisfy  the  change  objec¬ 
tives. 

XRB  analysts  directed  the  three  groups  (one  for  each  LAG)  to  generate 
shortfall  definitions  that  are  all  problem  oriented  (i.e.,  what  is  wrong). 

XRB  also  asked  them  to  generate  "fixes"  for  each  shortfall.  This  would  be  fol¬ 
lowed  by  having  them  develop  alternative  approaches  from  these  "fixes",  and 
change  objectives  from  the  shortfalls.  The  change  objectives  and  alternative 
approaches  would  then  be  matched/modified  as  required.  XRB  felt  that  the  devia¬ 
tion  from  the  original  process  of  identifying  shortfalls,  change  objectives,  and 
alternative  approaches,  in  that  order,  was  necessary  to  get  effective  ALC  parti¬ 
cipation.  It  was  found  that  the  ALC  planners  would  address  shortfalls  and  fixes 
but  had  some  difficulty  with  defining  change  objectives  and  alternative  approaches 
To  expedite  the  planning  the  ALC  proposals  were  directed  to  identify  fixes  cor¬ 
responding  to  each  shortfall. 

This  deviation  from  the  original  direction  does  not  appear  to  be 
serious  from  the  point  of  view  of  identifying  current  shortfalls.  However,  this 
would  tend  to  focus  the  planners'  attention  on  existing  shortfalls  and  not  on 
shortfalls  that  might  exist  in  the  future.  Whether  or  not  this  was  a  contribu¬ 
ting  factor,  the  resulting  shortfalls  and  "fixes"  have  tended  to  be  dominated 
by  existing  problems  rather  than  potential  future  problems.  Many  other  factors 
could  have  contributed  to  this  result  but  there  remains  a  problem  in  how  to 
elicit  future-oriented  ideas.  One  possible  solution  is  to  assure  the  partici¬ 
pation  of  at  least  a  few  senior  management  persons  in  each  planning  session  to 
provide  a  long-term  view  of  management  system  needs. 

The  actual  evaluation  of  alternative  approaches  can  be  performed 
through  a  variety  of  techniques.  The  techniques  described  earlier  appeared  to 
work  well  and  the  participants  seemed  to  be  comfortable  with  the  procedure. 
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In  fact,  they  were  apparently  able  to  continue  on  their  own  with  the  evaluation 
of  other  alternative  approaches  after  some  instruction  in  going  through  the 
evaluation  of  two  approaches. 

The  main  point  to  be  made  here  is  that  there  is  no  one  "correct"  or 
"best"  evaluation  technique.  It  is  probably  best  to  stay  away  from  numerical 
scoring  to  prevent  people  from  doing  mental  arithmetic  and  attach  undue 
meaning  to  what  are  in  fact  subjective  judgements.  The  purpose  of  the 
evaluation  procedure  is  to  help  structure  the  evaluation  process,  to  assure 
that  the  major  decision  factors  are  considered,  and  to  arrive  at  a  relative 
ranking  of  the  alternative  approaches. 

Conclusions 


The  LMS  Requirements  Determination  Planning  methodology  developed 
under  Contract  No.  F33600-80-C-0414  has  been  applied  and  adapted  to  LMS  planning 
for  the  Improvement  Process.  This  brief  report  covers  only  the  lessons 
observed  from  those  activities  that  Battelle's  Columbus  Laboratories  supported. 

Basically  the  planning  methodology  was  applied  much  as  planned  with 
adaptations  being  made  to  accommodate  the  TeleTeach  sessions.  The  lessons 
learned  that  are  generally  applicable  have  been  reflected  in  the  latest 
version  of  the  planning  methodology.  This  methodology  is  to  be  documented  in  a 
three-volume  "Handbook  for  Logistics  Management  Systems  (LMS)  Requirements 
Determination  Planning  Process",  scheduled  for  June  30,  1982. 
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